A Unified Fuzzy Data Model: Representation and Processing

A novel fuzzy data representation model which enables data mining with standard tools is introduced. Many data elements in the world are fuzzy in nature. There is an obvious need to represent and process such data effectively and efficiently, using the same standard tools for crisp data that are popular with researchers and practitioners alike. Currently, however, standard tools cannot process or analyze data that are not adequately represented. The comprehensive data representation model put forward here extends principles of binary databases and provides a unified approach to all types of data: discrete and continuous, crisp and fuzzy. The model is illustrated on a baseline dataset and tested in clustering experiments matched against controlled groupings and a real dataset. The tests confirm that the implementation of the model not only enables the use of standard tools but also yields better results as regards segmentation and clustering of fuzzy datasets. semantic models described in the background section, are unsuitable as input to standard clustering and mining tools which require one flat file matrix-like format such as a relational database table where each cell contains a single crisp value. The current paper proposes a method that enables the use of standard clustering and mining tools on fuzzy data. Since clustering has become increasingly popular as a data mining technique (Giannotti & Pedreschi, 2008; Manying, 2007) we concentrate on showing how our fuzzy data model applies to clustering. Clustering is crucial in the social sciences, marketing, finance, computer science, biology, medicine and elsewhere. Standard data analysis and mining tools such as SPSS, SAS or Clementine implement widely available clustering algorithms methods, and therefore are preferable DOI: 10.4018/jdm.2012010104 Journal of Database Management, 23(1), 78-102, January-March 2012 79 Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. over proprietary tools (for a brief description of clustering issues see Estivill-Castro & Yang, 2004; Gan, Ma, & Wu, 2007; Jain & Dubes, 1988; Jain, Murty, & Flynn, 1999; Lim, Loh, & Shih, 2000; Zhang & Srihari, 2004). There is an ongoing effort to devise a data model to resolve the discrepancy between the format used to store the database and the representation format demanded by clustering algorithms (Ryu & Eick, 2005). The few studies that have dealt with clustering of fuzzy data have been partially successful but the clustering methods were restricted to specific fuzzy data types and used dedicated and proprietary algorithms as described in the background section. The current paper proposes a unified data representation model that can deal with both crisp and fuzzy data, thus enabling the use of standard clustering and mining tools. The model draws on principles of binary representation employed in commercial databases, motion databases and fuzzy databases (Gelbard & Meged, 2008; Gelbard & Spiegler, 2002; Spiegler & Maayan, 1985). In these binary database models, the data are represented in a matrix where the rows stand for the database entities and the columns stand for different attribute values. In the proposed model, matrix cells are numbers that indicate degrees of attribute value similarity to the “right value”. These similarity numbers are derived from fuzzy database models such as the possibility distribution model (Prade & Testemale, 1984) and the proximity-based fuzzy relational model (Shenoi & Melton, 1989). The model is described and illustrated on a simple baseline dataset. It is tested by a controlled clustering experiment using SPSS software.